Convergent and Divergent Validity Documentation Based on Relations Among Content Areas
To examine convergent and divergent validity evidence, (a) correlations were were conducted to examine the relation between scale scores in the different content areas (ELA with Math, ELA with Science, and Math with Science), and (b) Chi-squared tests of independence were conducted to examine whether performance level scores (i.e., Advanced, Proficient, Emerging, of Novice) were independent across content areas at the p = .01 significance level. Note that the p-values were computed for a Monte Carlo test (Hope, 1968) with 2,000 replicates.
6.1 Correlations Among Content Areas
Correlations for ELA with Math ranged from 0.29 to 0.71, correlations for ELA with Science ranged from 0.61 to 0.82, and correlations for Math with Science ranged from 0.32 to 0.61. Please see Table 6.1.1. Across grades, correlations are almost always higher for Tier 1 than Tier 2 (except Grades 8 ELA with Math, and all Grade 11 correlations). Across Grades 4, 8 and 11, correlations for ELA with Science were higher than correlations for Math with Science.
Table 6.1.1. Correlations between Content Areas by Grade and Tier
|
Grade
|
Tier
|
ELA with Math
|
ELA with Science
|
Math with Science
|
|
3
|
1
|
.51
|
–
|
–
|
|
3
|
2
|
.51
|
–
|
–
|
|
4
|
1
|
.69
|
.82
|
.61
|
|
4
|
2
|
.42
|
.70
|
.40
|
|
5
|
1
|
.71
|
–
|
–
|
|
5
|
2
|
.39
|
–
|
–
|
|
6
|
1
|
.69
|
–
|
–
|
|
6
|
2
|
.35
|
–
|
–
|
|
7
|
1
|
.71
|
–
|
–
|
|
7
|
2
|
.36
|
–
|
–
|
|
8
|
1
|
.36
|
.77
|
.37
|
|
8
|
2
|
.39
|
.66
|
.32
|
|
11
|
2
|
.29
|
.61
|
.37
|
|
11
|
1
|
.41
|
.80
|
.40
|
6.2 Chi-squared Tests of Independence for Performance Levels Among Content Areas
Across grades and tiers, all Chi-squared tests of independence were statistically significant, except Grade 4, Tier 2 ELA - Science. Thus, in general, performance level scores in one content area were not independent of performance level scores in another content area. Please see tables….
Table 6.2.1. ELA - Math Performance Levels Chi-squared Test of Independence by Grade and Tier
|
Grade
|
Tier
|
Chi-Square
|
p-value
|
|
3
|
1
|
258.83
|
< .01
|
|
3
|
2
|
89.37
|
< .01
|
|
4
|
1
|
831.69
|
< .01
|
|
4
|
2
|
113.63
|
< .01
|
|
5
|
1
|
533.11
|
< .01
|
|
5
|
2
|
145.75
|
< .01
|
|
6
|
1
|
572.22
|
< .01
|
|
6
|
2
|
170.04
|
< .01
|
|
7
|
1
|
513.27
|
< .01
|
|
7
|
2
|
181.48
|
< .01
|
|
8
|
1
|
223.29
|
< .01
|
|
8
|
2
|
161.59
|
< .01
|
|
11
|
2
|
95.57
|
< .01
|
|
11
|
1
|
123.39
|
< .01
|
Table 6.2.2. ELA - Math Performance Matrix for Grade 3 Tier 1
|
|
Math
|
|
ELA
|
Advanced
|
Proficient
|
Novice
|
Missing
|
|
Advanced
|
79
|
0
|
0
|
9
|
|
Proficient
|
448
|
7
|
0
|
28
|
|
Emerging
|
326
|
167
|
1
|
59
|
|
Novice
|
198
|
4
|
1
|
17
|
Table 6.2.3. ELA - Math Performance Matrix for Grade 3 Tier 2
|
|
Math
|
|
ELA
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
52
|
39
|
0
|
9
|
6
|
|
Proficient
|
6
|
28
|
0
|
11
|
7
|
|
Emerging
|
0
|
0
|
2
|
2
|
2
|
|
Novice
|
2
|
10
|
5
|
15
|
5
|
Table 6.2.4. ELA - Math Performance Matrix for Grade 4 Tier 1
|
|
Math
|
|
ELA
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
151
|
221
|
6
|
21
|
29
|
|
Proficient
|
32
|
152
|
10
|
70
|
33
|
|
Emerging
|
0
|
26
|
229
|
83
|
58
|
|
Novice
|
5
|
64
|
44
|
80
|
17
|
Table 6.2.5. ELA - Math Performance Matrix for Grade 4 Tier 2
|
|
Math
|
|
ELA
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
76
|
48
|
0
|
7
|
10
|
|
Proficient
|
22
|
57
|
6
|
23
|
14
|
|
Emerging
|
0
|
0
|
3
|
1
|
1
|
|
Novice
|
0
|
1
|
3
|
3
|
1
|
Table 6.2.6. ELA - Math Performance Matrix for Grade 5 Tier 1
|
|
Math
|
|
ELA
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
34
|
18
|
0
|
2
|
4
|
|
Proficient
|
114
|
149
|
21
|
63
|
43
|
|
Emerging
|
3
|
12
|
217
|
80
|
41
|
|
Novice
|
12
|
76
|
81
|
83
|
24
|
Table 6.2.7. ELA - Math Performance Matrix for Grade 5 Tier 2
|
|
Math
|
|
ELA
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
57
|
58
|
3
|
16
|
8
|
|
Proficient
|
26
|
73
|
35
|
69
|
15
|
|
Emerging
|
0
|
2
|
17
|
8
|
2
|
|
Novice
|
7
|
29
|
40
|
27
|
8
|
Table 6.2.8. ELA - Math Performance Matrix for Grade 6 Tier 1
|
|
Math
|
|
ELA
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
110
|
134
|
11
|
50
|
47
|
|
Proficient
|
8
|
51
|
29
|
46
|
13
|
|
Emerging
|
1
|
15
|
264
|
29
|
36
|
|
Novice
|
2
|
32
|
54
|
37
|
16
|
Table 6.2.9. ELA - Math Performance Matrix for Grade 6 Tier 2
|
|
Math
|
|
ELA
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
83
|
35
|
0
|
6
|
10
|
|
Proficient
|
127
|
118
|
13
|
63
|
29
|
|
Emerging
|
0
|
1
|
9
|
2
|
2
|
|
Novice
|
10
|
28
|
10
|
24
|
6
|
Table 6.2.10. ELA - Math Performance Matrix for Grade 7 Tier 1
|
|
Math
|
|
ELA
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
95
|
62
|
4
|
35
|
31
|
|
Proficient
|
26
|
60
|
36
|
63
|
29
|
|
Emerging
|
1
|
4
|
186
|
29
|
17
|
|
Novice
|
2
|
17
|
90
|
70
|
21
|
Table 6.2.11. ELA - Math Performance Matrix for Grade 7 Tier 2
|
|
Math
|
|
ELA
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
187
|
130
|
3
|
50
|
34
|
|
Proficient
|
53
|
97
|
4
|
60
|
19
|
|
Emerging
|
0
|
3
|
6
|
18
|
3
|
|
Novice
|
8
|
24
|
2
|
42
|
8
|
Table 6.2.12. ELA - Math Performance Matrix for Grade 8 Tier 1
|
|
Math
|
|
ELA
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
130
|
10
|
3
|
4
|
20
|
|
Proficient
|
147
|
16
|
7
|
1
|
20
|
|
Emerging
|
25
|
94
|
2
|
0
|
14
|
|
Novice
|
152
|
71
|
5
|
0
|
37
|
Table 6.2.13. ELA - Math Performance Matrix for Grade 8 Tier 2
|
|
Math
|
|
ELA
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
27
|
135
|
11
|
31
|
24
|
|
Proficient
|
22
|
110
|
28
|
72
|
31
|
|
Emerging
|
14
|
10
|
42
|
20
|
11
|
|
Novice
|
18
|
34
|
29
|
62
|
12
|
Table 6.2.14. ELA - Math Performance Matrix for Grade 11 Tier 2
|
|
Math
|
|
ELA
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
231
|
45
|
0
|
15
|
14
|
|
Proficient
|
45
|
30
|
0
|
16
|
7
|
|
Emerging
|
10
|
12
|
1
|
16
|
2
|
|
Novice
|
11
|
7
|
0
|
6
|
4
|
Table 6.2.15. ELA - Math Performance Matrix for Grade 11 Tier 1
|
|
Math
|
|
ELA
|
Advanced
|
Proficient
|
Novice
|
Missing
|
|
Advanced
|
46
|
123
|
1
|
5
|
|
Proficient
|
6
|
72
|
0
|
4
|
|
Emerging
|
1
|
266
|
0
|
42
|
|
Novice
|
0
|
65
|
0
|
3
|
Table 6.2.16. ELA - Science Performance Levels Chi-squared Test of Independence by Grade and Tier
|
Grade
|
Tier
|
Chi-Square
|
p-value
|
|
4
|
1
|
421.51
|
< .01
|
|
4
|
2
|
59.14
|
< .01
|
|
8
|
1
|
388.10
|
< .01
|
|
8
|
2
|
201.48
|
< .01
|
|
11
|
2
|
62.73
|
< .01
|
|
11
|
1
|
182.21
|
< .01
|
Table 6.2.17. ELA - Science Performance Matrix for Grade 4 Tier 1
|
|
Science
|
|
ELA
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
225
|
1
|
161
|
41
|
|
Proficient
|
96
|
3
|
152
|
46
|
|
Emerging
|
12
|
99
|
201
|
84
|
|
Novice
|
48
|
4
|
131
|
27
|
Table 6.2.18. ELA - Science Performance Matrix for Grade 4 Tier 2
|
|
Science
|
|
ELA
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
105
|
0
|
22
|
14
|
|
Proficient
|
44
|
1
|
60
|
17
|
|
Emerging
|
0
|
0
|
4
|
1
|
|
Novice
|
1
|
0
|
7
|
0
|
Table 6.2.19. ELA - Science Performance Matrix for Grade 8 Tier 1
|
|
Science
|
|
ELA
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
9
|
70
|
1
|
63
|
24
|
|
Proficient
|
0
|
42
|
9
|
107
|
33
|
|
Emerging
|
0
|
0
|
101
|
11
|
23
|
|
Novice
|
0
|
15
|
98
|
116
|
36
|
Table 6.2.20. ELA - Science Performance Matrix for Grade 8 Tier 2
|
|
Science
|
|
ELA
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
35
|
142
|
3
|
22
|
26
|
|
Proficient
|
9
|
156
|
4
|
55
|
39
|
|
Emerging
|
0
|
21
|
10
|
49
|
17
|
|
Novice
|
0
|
50
|
4
|
84
|
17
|
Table 6.2.21. ELA - Science Performance Matrix for Grade 11 Tier 2
|
|
Science
|
|
ELA
|
Advanced
|
Proficient
|
Novice
|
Missing
|
|
Advanced
|
32
|
215
|
39
|
19
|
|
Proficient
|
0
|
47
|
41
|
10
|
|
Emerging
|
0
|
20
|
16
|
5
|
|
Novice
|
1
|
15
|
8
|
4
|
Table 6.2.22. ELA - Science Performance Matrix for Grade 11 Tier 1
|
|
Science
|
|
ELA
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
12
|
116
|
0
|
40
|
7
|
|
Proficient
|
0
|
45
|
0
|
31
|
6
|
|
Emerging
|
0
|
70
|
52
|
131
|
56
|
|
Novice
|
0
|
27
|
1
|
37
|
3
|
Table 6.2.23. Math - Science Performance Levels Chi-squared Test of Independence by Grade and Tier
|
Grade
|
Tier
|
Chi-Square
|
p-value
|
|
4
|
1
|
893.76
|
< .01
|
|
4
|
2
|
137.03
|
< .01
|
|
8
|
1
|
479.75
|
< .01
|
|
8
|
2
|
296.37
|
< .01
|
|
11
|
2
|
197.10
|
< .01
|
|
11
|
1
|
376.10
|
< .01
|
Table 6.2.24. Math - Science Performance Matrix for Grade 4 Tier 1
|
|
Science
|
|
Math
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
129
|
1
|
48
|
10
|
|
Proficient
|
164
|
3
|
261
|
35
|
|
Emerging
|
14
|
98
|
149
|
28
|
|
Novice
|
61
|
4
|
162
|
27
|
|
Missing
|
13
|
1
|
25
|
98
|
Table 6.2.25. Math - Science Performance Matrix for Grade 4 Tier 2
|
|
Science
|
|
Math
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
81
|
0
|
14
|
3
|
|
Proficient
|
49
|
0
|
47
|
10
|
|
Emerging
|
3
|
0
|
9
|
0
|
|
Novice
|
11
|
1
|
20
|
2
|
|
Missing
|
6
|
0
|
3
|
17
|
Table 6.2.26. Math - Science Performance Matrix for Grade 8 Tier 1
|
|
Science
|
|
Math
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
8
|
109
|
73
|
231
|
33
|
|
Proficient
|
0
|
10
|
128
|
38
|
15
|
|
Emerging
|
0
|
1
|
2
|
13
|
1
|
|
Novice
|
0
|
3
|
0
|
2
|
0
|
|
Missing
|
1
|
4
|
6
|
13
|
67
|
Table 6.2.27. Math - Science Performance Matrix for Grade 8 Tier 2
|
|
Science
|
|
Math
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
11
|
34
|
4
|
29
|
3
|
|
Proficient
|
28
|
187
|
2
|
49
|
23
|
|
Emerging
|
1
|
40
|
7
|
54
|
8
|
|
Novice
|
3
|
91
|
3
|
73
|
15
|
|
Missing
|
1
|
17
|
5
|
5
|
50
|
Table 6.2.28. Math - Science Performance Matrix for Grade 11 Tier 2
|
|
Science
|
|
Math
|
Advanced
|
Proficient
|
Novice
|
Missing
|
|
Advanced
|
32
|
210
|
45
|
10
|
|
Proficient
|
1
|
53
|
34
|
6
|
|
Emerging
|
0
|
0
|
1
|
0
|
|
Novice
|
0
|
28
|
22
|
3
|
|
Missing
|
0
|
6
|
2
|
19
|
Table 6.2.29. Math - Science Performance Matrix for Grade 11 Tier 1
|
|
Science
|
|
Math
|
Advanced
|
Proficient
|
Emerging
|
Novice
|
Missing
|
|
Advanced
|
9
|
38
|
0
|
5
|
1
|
|
Proficient
|
3
|
216
|
50
|
229
|
28
|
|
Novice
|
0
|
0
|
0
|
1
|
0
|
|
Missing
|
0
|
4
|
3
|
4
|
43
|
